true mean function
ARobustFunctionalEMAlgorithmforIncomplete PanelCountData
Panel count data describes aggregated counts of recurrent events observed at discrete time points. To understand dynamics of health behaviors and predict future negative events, the field of quantitative behavioral research has evolved toincreasingly rely upon panel count data collected viamultiple self reports, for example, about frequencies ofsmoking using in-the-moment surveysonmobile devices. However, missing reports are common and present a major barrier to downstream statistical learning.
Calibration Bands for Mean Estimates within the Exponential Dispersion Family
Delong, ลukasz, Gatti, Selim, Wรผthrich, Mario V.
Calibration Bands for Mean Estimates within the Exponential Dispersion Family null Lukasz Delong Selim Gatti Mario V. W uthrich Version of October 8, 2025 Abstract A statistical model is said to be calibrated if the resulting mean estimates perfectly match the true means of the underlying responses. Aiming for calibration is often not achievable in practice as one has to deal with finite samples of noisy observations. A weaker notion of calibration is auto-calibration. An auto-calibrated model satisfies that the expected value of the responses for a given mean estimate matches this estimate. Testing for auto-calibration has only been considered recently in the literature and we propose a new approach based on calibration bands. Calibration bands denote a set of lower and upper bounds such that the probability that the true means lie simultaneously inside those bounds exceeds some given confidence level. Such bands were constructed by Yang-Barber (2019) for sub-Gaussian distributions. Dimitriadis et al. (2023) then introduced narrower bands for the Bernoulli distribution. We use the same idea in order to extend the construction to the entire exponential dispersion family that contains for example the binomial, Poisson, negative binomial, gamma and normal distributions. Moreover, we show that the obtained calibration bands allow us to construct various tests for calibration and auto-calibration, respectively. As the construction of the bands does not rely on asymptotic results, we emphasize that our tests can be used for any sample size. Auto-calibration, calibration, calibration bands, exponential dispersion family, mean estimation, regression modeling, binomial distribution, Poisson distribution, negative binomial distribution, gamma distribution, normal distribution inverse Gaussian distribution. 1 Introduction Various statistical methods can be used to derive mean estimates from available observations, and it is important to understand whether these mean estimates are reliable for decision making. A statistical model is said to be calibrated if the resulting mean estimates perfectly match the true means of the underlying responses. In practice, calibration is often not achievable, as estimates are obtained from finite samples of noisy observations.
A Functional EM Algorithm for Panel Count Data with Missing Counts
Moreno, Alexander, Wu, Zhenke, Yap, Jamie, Wetter, David, Lam, Cho, Nahum-Shani, Inbal, Dempsey, Walter, Rehg, James M.
Panel count data is recurrent events data where counts of events are observed at discrete time points. Panel counts naturally describe self-reported behavioral data, and the occurrence of missing or unreliable reports is common. Unfortunately, no prior work has tackled the problem of missingness in this setting. We address this gap in the literature by developing a novel functional EM algorithm that can be used as a wrapper around several popular panel count mean function inference methods when some counts are missing. We provide a novel theoretical analysis of our method showing strong consistency. Extending the methods in (Balakrishnan et al., 2017, Wu et al. 2016), we show that the functional EM algorithm recovers the true mean function of the counting process. We accomplish this by developing alternative regularity conditions for our objective function in order to show convergence of the population EM algorithm. We prove strong consistency of the M-step, thus giving strong consistency guarantees for the finite sample EM algorithm. We present experimental results for synthetic data, synthetic missingness on real data, and a smoking cessation study, where we find that participants may underestimate cigarettes smoked by approximately 18.6% over a 12 day period.